Operational Analytics Data Management Systems

Prior to mid-2000s, the space of data analytics was mainly confined within the area of decision support systems. It was a long era of isolated enterprise data warehouses curating information from live data sources and of business intelligence software used to query such information. Most data sets were small enough in volume and static enough in velocity to be segregated in warehouses for analysis. Data analysis was not ad-hoc; it required pre-requisite knowledge of underlying data access patterns for the creation of specialized access methods (e.g. covering indexes, materialized views) in order to efficiently execute a set of few focused queries. The last decade witnessed a rapid overhaul in the area of business analytics. With the advent of ubiquitous data sources resulting in unprecedented explosion in ingestion volumes, analytic database systems had to evolve on multiple fronts. They were now required to provide high performance query processing over large volumes of data, handle adhoc queries, scale with the growing data volumes, excel in performance on clusters of commodity hardware, and last but not the least, capture very specific real-time analytic insights in live mainstream production environments. The decade long evolution of analytic databases has been paved with several technical milestones. Early to mid 2000s witnessed the emergence of MPP OLAP appliances (e.g. Teradata, Netezza, Exadata, Exasol) along with the resurgence of columnar data models (e.g. Actian Vector, Vertica) that were both capacity and compute-friendly. These appliances were multi-server systems with hundreds of computing cores and terabytes of storage. They came with integrated database management software that provided high performance query throughput on large volumes of data typically at rest. The same period also witnessed the dramatic rise of social and mobile applications that began generating volumes of unstructured raw data. Software frameworks such as Mapreduce and Hadoop paved the way for a new generation of analytic data management systems that batch-processed vast amounts of at-rest data (multi-terabyte data-sets) in-parallel on large clusters (thousands of nodes) of commodity hard-